Andrew Ng unveils vLLM course with Red Hat teaching KV cache memory management techniques in transformer model serving and history and technical architecture of vLLM LLM inference engine for 70B models.

According to @gdb, inference is the most valuable emerging software category and compute will increasingly be spent drawing samples from models, signaling a shift of compute budgets toward LLM inference workloads; source: @gdb on X, Nov 17, 2025. According to @gdb, OpenAI is inviting candidates to email gdb@openai.com for its inference team and to detail exceptional team accomplishments plus domain expertise in inference or large-scale system optimization; source: @gdb on X, Nov 17, 2025. According to @gdb, priority optimization areas include deeply understanding and optimizing the model forward pass, system-level efficiencies such as speculative decoding, KV offloading, and workload-aware load balancing, and managing and making observable a massive fleet at scale; source: @gdb on X, Nov 17, 2025. According to @gdb, this explicit emphasis on inference scaling provides a concrete data point for traders tracking AI infrastructure demand and its implications for serving efficiency and throughput in LLM inference; source: @gdb on X, Nov 17, 2025.